Optimizing for interpupillary distance in augmented reality eyewear

To create an accurate field of view (FoV), augmented reality (AR) eyewear is optimally matched to the user’s interpupillary distance (IPD). However, the IPD varies from person to person. This disclosure describes AR eyewear that (with user permission) automatically measures the IPD of a user and adjusts its depth and width to match the user’s IPD. The user facing interface is personalized based on the physical FoV of the user’s eyes

Meeting Summarization and Alerts for Video Conference Participants

A technique is proposed for summarizing meetings to help distracted participants transition back into conversations during video conferences. Processing logic may receive, from a video conferencing platform, a video file including multiple frames and audio. The processing logic may determine a period of time where a participant is distracted based on the participant’s body movement, eye movement, and camera settings. This determination can be made using a machine learning model. The processing logic can also generate a summary of the meeting based on received voice input, chat comments, and participant expressions. This summary may be generated using a generative artificial intelligence model. The processing logic can send a summary of the meeting to a participant for the time period during which they were distracted. This results in helping distracted participants integrate back into the conversation more easily during a video conference

Personalized Spatial Advertising in Augmented Reality (AR)

Augmented reality technologies can augment physical artifacts with interactive digital capabilities, accessible via various user devices. This disclosure describes techniques that enable personalized advertising based on interactive AR experiences. With user permission, a spatial element is added to personalized advertising by connecting it with the physical spaces or products involved in AR interactions. The content of the advertisement can be personalized to relevant aspects of the user’s characteristics and context, determined based on user-permitted data. Personalization can be achieved by associating physical objects and users with spatial anchors that capture relevant parameters of AR interactions. The personalized AR experience of the advertisement can be designed to enable users to purchase the advertised item at an online or physical location of the manufacturer and/or retailer

Revisiting Localized 3D Photographs Using Augmented Reality

This disclosure describes techniques that couple panoptic reconstruction with image localization and tracking to enable users to revisit their photographs in three-dimensional augmented reality at any time after the photo is taken. A user that takes a two-dimensional photo of some scenery, e.g., a city skyline, and then, years later, revisits the spot where the photo was taken is offered a user interface, e.g., a virtual slider in the field-of-view to see a before and after comparison of how the scenery has changed since the user’s last visit. Based on location similarity, the current view is matched, and the user is provided guidance to adjust their position to obtain a close match between the photograph and their current view

Using Code Review Repositories and Changelists to Train Large Language Models for Code Generation

While large language models (LLMs) can generate code, training of such models has not made use of data generated during the collaborative code review process that is a standard part of software development. This disclosure describes techniques that utilize historical code review data (including reviewer comments and corresponding code edits) available within organization internal code repositories to train LLMs to generate code. The historical code review data can be used for model tuning, to train an LLM via reinforcement learning from human feedback (RLHF), and/or via prompt engineering. The trained model can be utilized to generate code starting from code description provided using a prompt template. The prompt template can incorporate organization specific factors such as developer guidelines, developer or team style, etc. Code generated by the LLM can be iteratively refined via human review as well as from analytical tools that ensure style compliance, code coverage, test success rate, comment conventions, etc

Personalizing Audio Content Played While On Hold During a Phone Call

Calls made to a business by a customer, e.g., to request support, are often put into a queue waiting for a human agent to be available. During the hold time, canned music or other audio is played back to the caller. Such audio is low quality owing to the limited capacity of the telephony channel, is not personalized, and repeated multiple times till an agent becomes available, providing an unsatisfactory calling experience. This disclosure describes the use of machine learning techniques to detect canned audio and replace it with high fidelity music or other content. With user permission, the replacement content can be personalized, e.g., based on a user’s music playlists/preferences, and context. Machine learning techniques can also be utilized to upscale music on hold experience provided by the business. With user permission, advertising content or helpful content about the business can be delivered during the hold time. The techniques can be integrated into a virtual assistant or device operating system to provide an improved calling experience

SONG SENTIMENT RECOGNITION BASED ON IMPLICIT USER FEEDBACK

Computing devices utilize user feedback (e.g., explicit and/or implicit feedback) to determine a user’s sentiment towards a song or music played by the computing device or another speaker within a given environment. The computing device may determine the user’s sentiment, emotion, reaction, or opinion towards a song and utilize that information when selecting music, tunes, or songs to include in a playlist or otherwise output to the user. In one example, to determine the user’s sentiment towards the song, the computing device executes a sentiment recognition algorithm that utilizes implicit feedback, such as a user singing along with the song, the user moving in time with the beat of the song, etc., which may be captured by one or more sensors of the computing device. In some instances, the computing device may also use explicit feedback provided by the user, such as selecting a “like” or a “dislike” button to determine the user’s sentiment towards the song. If the user enjoys the song, the computing device may recommend, suggest, or output the song or similar songs (e.g., similar artists, genres, etc.) for playback, for example, when selecting songs to be included in an autogenerated playlist

User Customizable Content Summarization Using Deep Learning

Users expend substantial time and mental effort in trying to distill the most important facts and/or insight from lengthy text content. While deep learning models can summarize long documents, such models typically operate on the backend. Model output is not customizable by end users via simple user interfaces. This disclosure describes mechanisms that provide users with the ability to obtain customized automatic summaries of documents. Users can specify the desired properties of the summaries based on their preferences and constraints such as available time, desired length or style for the summary, output language, complexity, etc. via convenient UI elements. Users can also choose to obtain automatically generated images or other media that summarize the document content. Customization of output can make the automatically generated summary more relevant and useful and save the user time and effort in reading lengthy text

Improving Remote Customer Interaction Experiences Using Machine Learning

A common problem in contact centers is high employee turnover. Artificial intelligence (AI) techniques that have been introduced to smoothen interaction and improve the customer’s experience can have opposite effects, e.g., by requiring the customer to navigate complex menu options. This disclosure describes AI-based techniques applied to agent training and customer calls. The techniques can reduce turnover at contact centers and improve the experience of end users who interact with customer service agents. Per the techniques, suitable AI techniques are implemented to train human customer agents, and human feedback is in turn used to train AI techniques. Human-AI augmentation can be used to mirror the communication styles of customers to improve the interaction experience. The techniques can also be used to improve safety, e.g., by automatically detecting scam calls and alerting users. The techniques enable the creation of scalable, standalone, artificial or human-AI augmented customer service agents

AI-based Image Synthesis for Enriched Search and Shopping

Across numerous applications, notably in search and shopping for unique items, humans are constrained by what has already been built or designed. This disclosure describes techniques that leverage natural language-based, deep-learning image synthesis to deliver enhanced product search via services such as search engines or e-commerce websites. The synthetically generated products can be custom manufactured upon order. Unconstrained by real world objects, the techniques deliver to the search engine or e-commerce user synthetic objects based on text descriptions provided by the user